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ABSTRACT 

This library evaluation is based primarily on 
responses of some 320 members of the professional research staff and 
50 members of the professional administrative staff at the Langley 
Research Center to a questionnaire. Evaluation first considers the 
library as a whole as only one part of a total system of information 
sources serving professionals. This is followed by an evaluation of 
the collection, facilities and tools, and services of the library 
based on responses of those who use the library more than six times a 
year. A major feature of the evaluation is the use of a critical 
incident technique in which respondents furnish details of a recent 
incident when they located information which proved useful in their 
work. The role of the library is then inferred from comparison of 
information source utilization using a previous study as a standard. 
The analysis of the library collection, tools and facilities, and 
services provides the basis for more detailed evaluation and 
recommendations. (Author /SJ) 
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THE ROLE OF THE LRC TECHNICAL LIBRARY 



IN FULFILLING THE INFORMATION NEEDS 
OF PROFESSIONAL EMPLOYEES 
H. Dudley Dewhirst* 

ABSTRACT 

This library evaluation is based primarily on responses of some J20 
members of the professional research staff and 50 members of the professional 
administrative staff to a questionnaire. Evaluation first considers the 
library as a whole as only one part of a total system of information sources 
serving professionals. This is followed by an evaluation of the collection, 
facilities and tools, and sendees of the library based on responses of 
those who use the library more than six times a year. 

A major feature of the evaluation is the use of a critical incident 
technique in which respondents furnish details of a recent incident when they 
located information which proved useful in their work. The detailed method- 
ology closely follows that used by Rosenbloom and Wolek^ in a major study of 
information sources used by scientists and engineers in four large corpora- 
tions. The role of the library is then inferred from comparison of 
information source utilization using the previous study as a standard. 

The analysis of the library collection, tools and facilities, and 
services provides the basis for more detailed evaluation and recommendations. 

*Assistant Professor of Industrial Management, University of Tennessee, 
Knoxville, Tennessee. 

Rosenbloom, Richard S. , and Wolek, Francis W. : Technology, Information 

and Organizatio n. Boston: Harvard Graduate School of Business Administration, 
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INTRODUCTION 



This report is the result of a 10-week research project conducted as 
part of the 1970 ASEE-NASA Summer Institute. The project was an evaluation 
from the user's point of view of the LRC Technical Library. The primary 
sources of information for the evaluation were two sets of questionnaires. 

The first, questionnaire was directed to members of the professional research 
staff in the research divisions, engineering and technical service divisions, 
and project offices. The second questionnaire was sent to a smaller group 
of professional -level personnel in administrative divisions. 

The responses to these questionnaires, supplemented by information 
furnished by the library staff and the observations by the author provide 
the basil for the evaluation. 

Section I is a brief discussion of what has been called the "information 
explosion" and the library's response to a dramatic increase in the size and 
scope of its collection. 

This is followed by a discussion of methodology in Section II which 
explores the difficult problem of library evaluation, explains the strengths 
and weaknesses of the approaches used. This portion of the report provides 
a description of sampling technique and a profile of the respondents. 

Section III reviews the role of the library as part of tlis total system 
of information sources available to users. Reported usage as well as 
importance rankings of the library and other information sources are com- 
pared. In addition, data on information source utilization from Langley 
engineers and scientists are compared with similar data obtained in 
Technology, Information and Organization , a major study by Richard Rosenbloom 
and Francis Wolek. 
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Section IV reports and analyzes the responses of users of the library 
to questions pertaining to the collection, the services offered, and the j 

■ i 

tools and facilities offered by the library. j 

Section V presents the conclusions reached, makes recommendations, and 
presents several questions about the long-term policy with respect to 
information services provided by Langley for its professionals. , 

The author would like to acknowledge the generous help of the entire 
library staff, without which this report would not be possible. In J. 

j 

particular, my thanks go to Phil Weatherwax, Jane Hess, and Willaree Curtis > 
for answering innumerable questions about library operations; to Betty j 

Gilman for her help in literature searching; to Dilsey Hawthorne for her j 

aid in coding and keypunching data from the questionnaires; to Marshal f 

| 

Hughes II for his programing and begging of computer time; and to Oneda 
Moore for exceptionally quick service in typing drafts of this report. 

i 
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SECTION I 



THE DATA EXPLOSION: IMPACT AT LANGLEY 

"Data are the very latest kind 
of pollution." Stafford 3eer 



The task of the LRC Technical Library and, in fact, every scientific 
or technical library has been made more difficult because of dramatic 
increase in the amount of material published. Kosmetsky suggests the magni- 
tude of the explosion in the following: 

Let me try to relate the data e:q?losion to the mount of reading 
one would have to do weekly in order to keep current with tech- 
nology through published works. In 1900 the weekly stack of 
published material would be 5 feet high, 1 foot wide, and 1 foot 
long. In i960 the stack would be 5 feet high, 1 foot wide, and 
60 feet long. Predictions have shown that by the year 2000, the 
stack will be 5 feet high, 15 feet wide, and 60 feet long. 

(Kosmetsfcy, 1970# P* IO5. ) 

While Kosmetsky' s demonstration spans a long time period, it amounts 
to an approximate doubling of data every 10 years. While such a growth rate 
is dramatic in itself, it does not equal the rate at which the total collec- 
tion at the LRC Technical Library has grown in the last decade. 

Table 1-1 indicates the approximate size of the collection at Langley 
for the years i960, 1965, and 1970* The data are estimates since data have 
not been kept in this form in the past. 

It can be seen that growth has been dramatic in all areas. Over the 
last decade the report collection has increased by 167 percent, the 
periodical collection by 147 percent, and the book collection by nearly 
200 percent. 
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TABLE 1-1.- SIZE OP COLLECTIONS IN THOUSANDS 1960-1970 



Collection 


Year 


I960 


1965 


197Q 


Reports: 








Paper cop:/' available locally 


1 120 


10 O 


120 


Paper copy - other NASA Centers 


50 


— 


— 


Microfiche (excluding locally available 
paper duplicates) 


0 


100 


180 


Machine indexed (not available locally) 


0 


20 


100 


Total reports 


150 


220 


4oo 


Periodical articles: 








Bound periodicals^ 


300 


360 


660 


Microfiche (excluding locally available 
paper duplicates) 


0 


5 


40 


Machine indexed (not available locally) 


0 


20 


40 


Total periodical articles 


300 


385 


740 


Books: 

. 


13 


24 


38 



^Article content of bound periodicals estimated at 
So articles per bound volume - 



Examination of table 1-1 demonstrates changes in the nature of the 
collection as well as growth, in all portions of the collection. In the 
reports sector, aj 1 of the growth has occurred in two new forms of data 
storage, namely, microfiche and access to data stored at a centralized 
information center located at College Park, Maryland. 

With the establishment of the centralized facility at College Park, 
Maryland, the cataloging of documents was substantially eliminated at 
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Langley. The College Park facility performs abstracting and cataloging for 
the entire NASA system. A key feature of this centralized system is a 
computerized information retrieval system which serves two basic functions. 
First, master index tapes are provided to major participating libraries. 
These tapes can be used to provide an index by report number, author, and 
subject. Secondly, participating libraries can, through use of a remote 
console (RECON), search the entire stored data collection by using a system 
of key words to find reports on a particular subject. 

By curtailing its cataloging function, utilizing microfiche (also 
provided by the College Park facility), and using indices prepared from 
master tapes provided by the facility, the LRC library has managed to keep 
up with the data explosion and serve a greater number of patrons with a 
workforce smaller than that of 10 years ago. These data are shown in 
table 1-2. 

TABLE 1-2.- GROWTH IN CIRCULATION PATRONS AND STAFF, 1960-1970 





i 960 


1970 


Percent increase 
(decrease) 


Patrons* 


1,100 


1,600 


45 


Monthly circulation 


2,500 


5,500 


no 


Staff 


26 


22 


( 15 ) 



*Approximate size of professional staff (includes 
all employees in classification 700 - Aero-Space 
Technologist ) . 



In addition to its sharp growth, the pattern of circulation has changed. 
Since 1964- the proportion of circulation attributable to reports has dropped 



from 77 percent to 56 percent; book circulation has remained about the same 
at 10 to 11 percent; periodical circulation has increased from 13 percent to 
33 percent. The changes in the size and nature of the collection, changes 
in library functions, and changes in the tools and techniques used have 
made the LRC Technical Libraiy a different library from the one that went 
under the same name 10 years ago. 
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SECTION II 



AN APPROACH TO THE PROBLEM OF EVALUATION 



"Any evaluation must incorporate 
feedback from the user." F. L. Scheffler 



The evaluation of any library is a difficult task for numerous reasons . 
There are several approaches, none of which is completely satisfactory. The 
present study uses a combination of several approaches with emphasis on the 
user's viewpoint. 

Briefly summarized, approaches to library evaluation fall into one of 
several types as follows: 

A. Economic - Costs are compared to those of other libraries^ either 
on a total or unit cost basis to determine how efficiently the library 
.carries out its function. 

In many cases, in conjunction with the cost study, an 
attempt is made to estimate the value of the library to the user. The 
problem with this approach is that "the value of information and its impact 
on the ultimate research performed almost defy mensuration" (Scheffler and 
March, 1970, p. 3)- Since costs by themselves (i.e., without the resulting 
benefits) are not a meaningful criteria of performance for the library's 
function - the providing of useful information - this approach was not used 
to any significant degree in this study. 

B. Observation and Analysis of Operations - This method relies on 
judgments formed by observing operations and analysis of statistical data 
such as circulation. This evaluation utilizes this approach to a limited 
degree. 




" "a 

ERIC 



— \ •• V 



13 



7 



C. User Survey - This method is based on the premise that service to 



the patron is the most important criteria on which to judge a library, rt 
assumes that users, collectively, axe reasonably good judges of the value 
of a library. This report is based primarily on a survey of users. A 
questionnaire approach was used in order to obtain a sufficiently large 
sample in a limited time period. 

There are different types of questions which can be 
asked of library patrons. One type of question is the opinion question 
(e.g., Is the library adequately fulfilling your needs? ) One major problem 
with this approach is that some respondents perceive ulterior motives for 
the question and respond accordingly. Some may feel library personnel will 
"catch it" if they respond unfavorably, while others may feel the library's 
budget will be cut if the evaluation is too glowing. Some respondents will 
furnish the answers they think the researcher wants, with, little regard for 
their validity. In any event , the results are likely to be biased, and 
determining the direction and magnitude of the bias is not an easy task. 
Because of these difficulties, this survey makes only minor use of opinion- 
type questions. 

Another approach is to ask users how they behave . 
While this approach is not completely free from possible bias, it substan- 
tially reduces it. For this reason, most of the questions used in this 
survey ask about behavior rather than opinion. Most are specific, although 
there are several open-end-type questions. Use is made of a "critical 
incident" technique which asks the respondent to recall and give details 
of a recent instance in which he found information useful in his work. 
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This technique has received favorable comment from a number of researchers 
in the information use field (Menzel, 1966 , p. h- 1 ). 

Review of Fast Library Evaluations 

A review of library evaluations made by various researchers indicates 
some of the factors which must receive some consideration in any evaluation. 

First, a library is only part of a very complex system used by 
scientists and engineers to satisfy their information needs. Any evalua- 
tion of a library must, therefore, consider the total set of information 
sources (Paisley, 1968, p. 3 )- 

Secondly, it is not valid to suggest that all individuals should use 
the library to the same degree. It has been shown that engineers and 
scientists utilize the various information sources to a greater or lesser 
degree, depending on whether they are working on research tasks or a 
development/design-type tasks (Rosenbloom and Wolek, 1966). These authors 
suggest that the differences in task requirements cause a different set of 
information needs which can best be satisfied by using a different mix of 
information sources. 

Further substantiating the idea that not everyone should use the 
library, it has been demonstrated by Allen (1969) that there exist individuals 
within organizations who aid the flow of valuable information. These 
"technological gatekeepers" are (l) better acquainted with the scientific 
and technological literature and (2) maintain a greater degree .of informal 
contact with peers outside their organizations. These individuals occupy 
key positions in the communication network in the organization. They serve 
as consultants and information sources for others in their work groups. 
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Such a naturally occurring system may be the most economical means of 
keeping groups of individuals 'well informed about activities in their 
particular field of research. There is, in effect, a two-step communication 
process in which the gatekeeper spends his time in the library, thus 
reducing the need for others to do so. 

The above discussion indicates some of the reasons why the amount of 
usage of a library is an imperfect criteria for its evaluation. In addition 
to the difficulty of determining a standard for the "right amount" of 
utilization, there is also the problem of cause. For example, if a library 
is determined to be under-utilized, then is the fault in (a) the facilities 
for making the collection available to the user, (b) the collection itself, 
or (3) the potential user? 

In spite of these difficulties, utilization of the library is used as 
a primary criterion in the evaluation. Several steps were taken to control 
or alleviate the problems discussed above. First of all, it is assumed 
that engineers and scientists approach the task of gathering information 
in a sensible manner. It has been shown by Allen (1969) that use of an 
information source by engineers and scientists is a function of (l ) the 
perceived accessibility of the source and (2) the perceived technical 
quality of the source. Therefore, it is possible to infer from high volume 
of usage of an information source that the particular source is accessible 
and of good technical quality. 

Fortunately, there are several studies of excellent quality which can 
be used as a reference point for evaluation. One in particular, Technology, 
Information and Organization , by Richard S. Rosenbloom and Francis Wolek, 
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explored the utilization of various information channels by over 1,500 
scientists and engineers in four large corporations. Some 650 members of 
that sample are employed in central research divisions of their organiza- 
tions and provide a group comparable to LRC research professionals. These 
researchers explored differences in information source utilization as a 
function of academic discipline (scientist versus engineer), amount of 
education (bachelors versus advanced degree), length of service, type of* 
organization (central research laboratory versus operating division), and type of 
task (research versus development/design). 

This study at Langley Research Center utilized several of the same 
questions used in the Rosenbloom and Wolek report.^ - Thus, it is possible 
to compare information source utilization of LRC professionals with that 
of a large number of corporate engineers and scientists. Since all of 
these corporations have fairly extensive libraries, it is possible to com- 
pare the libraries by comparing the utilization of various information 
sources. The data developed by Rosenbloom and Wolek serve as a standard 
for information source utilization. 

Differences in background, type organization, and work task are con- 
trolled by comparing data from groups of LRC professionals with data from 
groups in the Rosenbloom and Wolek report of similar background, type 
organization, and work task. 

In addition to evaluating the library as part of a complex information 
system, this report also examines the respondents' usage of particular 

^ith the much appreciated permission of Dr. Richard S. Rosenbloom of 
the Harvard Graduate School of Business Administration. 



portions of' the library collections, use of help from the library staff, 
and use of library tools and services. Only respondents indicating a 
certain minimum amount of usage (more than six times for the last 12 months ) 
were asked to respond to the questions pertaining to details of library 
usage. Inferences are drawn from responses to these questions about the 
accessibility and usefulness of the various library collections, services, 
and tools. 

Up to this point, our discussion has centered on the use of the 
library by the research staff - professional-level engineers and scientists. 
However, in the past several years, the library has attempted to broaden 
its usefulness by expanding its collection in the management area. For this 
reason a second, shorter questionnaire was directed to a sample of professional - 
level personnel in the administrative divisions. 

The Questionnaires and Respondent Background 

A questionnaire consisting of 55 questions was designed for engineers 
and scientists engaged in research tasks at Langley. This questionnaire is 
presented in appendix A. 

The population selected for sampling included all JOO classification 
(Aero-Space Technologists) employees who met the following criteria: 

1. Had a minimum of 6 months' employment with NASA at Langley 
Research Center. 

2. Had obtained a bachelor's degree or more in science, engineering, 
or mathematics. 

5. Were a member of a research division, an engineering and technical 
service division, or a project office. (Hereafter, this group will be 
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referred to as the Professional Research Staff, even though it includes 
some members not in research divisions. ) 

A total of 458 members of this population was selected at random to 
receive questionnaires. Of these, eight were returned as undeliverable 
because of educational or sick leave, reducing the effective distribution 
to 450. Of these, 340 (or 7 6 percent) of those questionnaires actually 
delivered were completed and returned. Approximately 20 of these, however, 
were excluded from the analysis because they were returned too late. A 
summary of selected background characteristics of the respondents is 
presented in table II-l. 

A second questionnaire containing l8 questions was designed for 
professional -level administrative personnel. The population selected for 
sampling included all oOO' classification employees who met the following 
criteria: 

1. Had a minimum of 6 months' employment with NASA at Langley 
Research Center. 

2. Were a member of one of the divisions under the Assistant 
Director for Administration. 

3. Not assigned to the library or to the staff of the Assistant 
Director. 

A total of 75 questionnaires were distributed at random. Of these, 

50 (or 67 percent ) were returned} 49 were usable and are included in the 
data presented. 

A summary of background characteristics of the respondents is presented 
in table II-l. 
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TABLE II-l. - BACKGROUND CHARACTERISTICS OF RESPONDENTS 



Charact eri sti c 


Research 
staff 
N = 319 


Administrative 
staff 
N = 49 




Percent 


Age 






20 - 24 


6 


2 


25 - 29 


18 


8 


30-54 


24 


14 


35 - 39 


14 


16 


40 - 49 


29 


25 


50+ 




-22 




100 


100 


Length of service 






0-2 years 


3 


6 


3-5 years 


l4 


14 


6-10 years 


35 


37 


11 - 15 years 


19 


10 


15 - 20 years 


8 


6 


20+ years 


21 


SL 




100 


100 


Education 






High school 


0 


42 


Bachelor 


66 


48 ■ 


Master 


29 


8 


Fh.D. 




2 




100 


100 


Academic discipli.ne 






Science/mathematics 


• 32 


11 


Engineering 


68 


— 


Business 


— 


28 


Liberal arts 


— 


10 


Other . » ' 


— 


11 


No degree 


— 


4 o 




100 


100 


Supervisory level 






Individual contributor 


76 


64 


Group/tJnit/Section Head 
Branch, Ass't. Branch Head 


15 


18 


7 


10 


Division, Ass't. Division Chief 


2 


8 




■ " ■ 


■ — 




100 


100 
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The Question of Bias 



Are the answers biased? Probably yes, although the degree of bias 
does not seem excessive. Two checks were made to determine the amount of 
bias: 

1. A check was made to determine if those vmo responded supplied 

reasonably reliable answers. This was done by comparing the frequency of 

) 

library copy service indicated by respondents (see Question 28, appendix A) 

A 

with recent statistics on copies provided to all patrons. Assuming that 
the respondents are a representative sample, then the comparison of 
respondents' estimates to the act ual number of copies distributed indicates 
that respondents' estimates are 20 percent high. However, part of this 
bias is apparently a result of non-respondent bias (explained in the 
following paragraph) rather than inaccurate answers on the part of those 
who responded. 

2. A second check was made by comparing the percentage return of 
questionnaires from various divisions with the theoretical response that 
would occur if the questionnaires were distributed in proportion to the 
professional research staff strength (as is highly probable with the random 
selection system utilized to draw the sample). Those divisions with 
particularly high or ‘particularly low returns were checked to determine if 
they also had a particularly high or low indicated percentage of library 
users. If this were true, then non-respondent bias (e.g., library users 
tend to respond, non-users do not) would be indicated. It was noted that 
the project office personnel had both a low respondent percentage return 
(38 percent) and a low percentage of library users (42 percent). However, 



personnel in all other divisions appeared to respond without regard to the 
degree of library usage. Since project office personnel represent less 
than 8 percent of the total professional research staff, it was concluded 
that their general lack of response would not unduly bias the results. 

It does appear, therefore, that the data may have a bias in favor of 
the library. However, the bias does not appear to be of such a siae as 
to significantly change the basic conclusions of the report. 
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SECTION III 



THE LIBRARY IN COMPARISON WITH OTHER INFORMATION SOURCES 

"We believe that the means by which 
scientists and engineers acquire 
information are such that, in their 
view, their behavior gets the job 
done . " Rosenbloom and Wolek 

In this section of the report, the library is considered as a whole 
and analyzed as one part of the complex system of information sources used 
by engineers and scientists. The library is evaluated in two ways. First, 
data on information sources resulting from this survey are compared with 
similar data from a major study of information sources of scientists and 
engineers (see Section II). Second, data on frequency of utilization and 
perceived importance of various information sources are compared. 

Also in this section there are comparisons of information source 

utilization among subgroups of Langley professionals divided according to 

degree level (bachelors versus advanced degrees), job task (research versus 

design and development), and length of service and divisional affiliation. 

Comparison of Information Sources; Langley Versus Other Research 
Organization s 

In their report, Technology, Information and Organization, Rosenbloom 
and Wolek (hereafter R and w) gathered data on about 65 0 scientists and 
engineers who were employed at central research laboratories of four large 
corporations. The present survey at Langley gathered data using the same 
questions with the same instructions as used in the R and W study (see 
Questions .11-1^ and related instructions in Questionnaire, appendix A). 

In these questions, respondents were asked to recall and give details of a 
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recent instance in which they found a technical idea or item of* information, 
from a source outside their immediate work. group, which proved useful in 
their work. 

A comparison of information source utilization by Langley engineers 
and scientists with that from the R and W study is presented in table III-l. 



TABLE III-l. - SOURCES OF USEFUL INFORMATION: LANGLEY RESEARCH 

PROFESSIONALS COMPARED WITH PROFESSIONALS IN CENTRAL 
RESEARCH LABS OF FOUR MAJOR CORPORATIONS 



Information source 



Percent 


Langley 
N = 272 


R and W 
N = 655 


42 


39 


8 


6 


_8 


10 


58 


55 


25 


28 


11 


R 


42 


45 



Written media : 

Outside professional literature 
(books, journals, reports by 
other organizations) 

Inside professional - in-house 
reports 

Trade - catalogs, trade publications, 
technical trade literature 



Total written 



Oral media: 



In-house - with members of own 
organization 

External - with employees of other 
organizations 

Total oral 



Comparison of the two indicates that Langley research professionals 
find slightly more useful information in professional literature than do 
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similar professionals queried in the R and W report. It should be noted 
that both these media are the stock in trade of the library. These data 
can be taken as one piece of evidence that the library is an excellent one 
because it has changed the normal pattern of information uses utilized. 
Gerstberger and Allen (1968, p. 279) have demonstrated that accessibility 
and perceived technical quality are the criteria by which research and 
development personnel decide to use a particular channel. Thus, the fact 
that Langley research professionals use the professional literature more 
than R and W research professionals indicates that the professional litera- 
ture at Langley is more accessible and/or is of higher technical quality 
than it was for the R and W professionals. Making professional literature 
of high quality accessible to users is, of course, the library's reason for 
existence. It would be erroneous to assume that the library provided 
50 percent (of the sum of the two professional literature categories) of 
all useful information. The library was, however, involved in the process 
often. 

The Langley survey asked (but, unfortunately, R and W did not) what 
role, if any, was played by the LRC library in the particular incident in 
which a useful item of information was found (see Question 15 ). .The 
responses are shown in table III-2. 
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TABLE III-2. - ROLE OF LRC LIBRARY IN' SUPPLYING USEFUL 
INFORMATION TO RESEARCH PROFESSIONALS 



Role of library 


Percent* 


Library personnel provided lead or found information 


6 1 


A lead was found in an abstract, book, report, or 


8 


article which respondent had obtained from the 
library 




Substance of information was found in book, report, 


8 


or article which respondent had obtained from the 
library 




Substance of information, or a lead was found in 


7 


document which someone other than respondent had 
obtained from the library 




Library was involved in more than one of the above 




ways 


& 



*Based on 272 responses. 



Of the 272 instances, the library was involved in either providing 
a lead, or finding the information, or providing a document in which a lead 
or the information itself was found. Thus, the library was involved, in 
one way or another, in 55 percent of the reported incidents of a respondent* 
finding useful information outside his immediate work group. That is 
exceptionally high for any single information source and demonstrates the 
major role played by the LRC library in supplying useful information. 
Comparisons of Subgroups Within NASA and Corporate Samples 

As pointed out earlier in Section II, information source utilization 
varies as a function of education (advanced degree holders use written 
sources more), job task requirement (research tasks require use of more 
written sources than do development and design tasks). For this reason. 



it was decided to divide the respondents into several subgroups and compare 
the results with similar data from the R and W study. Ry this procedure, 
it was felt that it would be possible to control some of these variables 
which influence information source utilization and, by making additional 
comparisons, give the data a chance to either reinforce or contradict the 
tentative conclusion reached that the LRC library has, through its excellence, 
changed the normal pattern of information source utilization. 

One of these comparisons of research professionals divided by job task 
is presented in table III-3- It should be noted that criteria for dividing 
respondents between research task and design/development task were substan- 
tially the same in both studies. 



TABLE III-3.- SOURCES OF USEFUL INFORMATION: COMPARISONS OF LANGLEY 

AND CORPORATE PROFESSIONALS DIVIDED BY JOB TASK 



Job task 


Information source 


Percent 


Langley 


R and W 


N = 125 


N = 211 




Written media: 








Outside professional 


46 


44 




In-house professional 


10 


5 




Trade 


4 


8 


Research 




So 


57 


task 










Oral media: 








In-house 


19 


21 




Outsider 


21 


22 






5o 


5 3 






N = 89 


N = 263 




Written media: 








Outside professional 


33 


33 




In-house professional 


7 


6 


Development 


Trade 


12 


11 


or design 




53 


50 


task 


Oral media: 








In-house 


24 


36 




Outsider 


22 


14 






W 


50 



21 
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Data show that the professional literature is more widely used by 
NASA research professionals than by other professionals located at central 
research laboratories engaged in similar tasks. The difference is only 
1 percent when comparing those engaged in development /design tasks, but is 
J percent when comparing those involved in research tasks. 

One further comparison between Langley research professionals and those 
in central research laboratories is possible. In this case the two samples 
are subdivided according to degree level. This comparison is shown in 
table III-4. 



TABLE III-4. - SOURCES OP USEFUL INFORMATION: COMPARISON OF LANGLEY 

AND CORPORATE PROFESSIONALS DIVIDED BY DEGREE LEVEL 



Education 


Information source 


Percent 


Langley 


R and W 


52$ 

11 

VO 

oo 


55$ 

II 

4=- 




Written media: 








Outside professional 


56 


42 




In-house professional 


9 


6 




Trade 


6 




Advanced 




71 


55 


degree 










Oral media: 








In-house 


12 


26 




Outside 


iX 


12 






29 


5 ? 






it 

H 

11 


N = 194 




Written media: 








Outside professional 


34 


33 




In-house professional 


8 


7 




Trade 


Jl 


l 6 


Bachelor ’ s 




51 


56 


degree 










Oral media: 








In-house 


24 


32 




Outside 


25 


12 






59 


55 



IER1C 



22 
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The most striking difference occurs when comparing Langley research 
professionals with advanced degrees with similarly educated professionals 
from the R and W study. Including both in-house and outside professional 
literature, the Langley group uses professional literature in 65 percent of 
the incidents, whereas the corporate professionals use it in only 48 per- 
cent of the reported incidents. While the comparison of sources of useful 
information between personnel with bachelor’s degrees does not result in 

4 

differences as large as those demonstrated by those with advanced degrees, 
once again Langley professionals are using the type of literature that the 
library deals in to a greater extent than other research professionals. 

There are other background factors or characteristics which were 
found to make a difference in information sources utilized by research 
professionals in the R and W study. It was found that scientists (as 
opposed to - engineers ) and those with less than 10 years’ service (as 
opposed to those with more than 10 years’ service) made greater use of the 
written and, particularly, professional literature sources of information.^ 
On both counts, the comparisons of Langley and the R and W data from 
the central research division should have indicated lower usage of written 
sources since the Langley staff is composed of more engineers (68 percent 
versus 26 percent) and more 10-year-plus employees (48 percent versus 
JO percent) than the R and W sample of central research laboratory 



1 These findings were based on R and W’s total sample of over 1,500 
which included scientists and engineers from operating divisions as well as 
the central research laboratories. For this reason, the data are not pre- 
sented and compared with data from NASA. 




29 



23 



personnel. Thus, the differences shown by Langley professionals in use of 
the library -supplied sources are even more impressive. 

If one accepts the premise that engineers and scientists use inform- 
ation sources on the basis of their accessibility and their quality, then 
it seems a reasonable inference that the IiRC library is doing an excellent 
job of making quality information sources highly accessible. 

Information Sources - Importance and Frequency of Use 

Questions 1 6 and 17 asked respondents to indicate the frequency with 
which they used various information sources and the degree of importance of 
each source to their work. In answering the importance question, respondents 
were asked to indicate the degree of importance by selecting one of five 
answers ranging from Veiy Low to Very High. The answers were scored as 
follows: Very Low = 0, Low = 1, Moderate = 2, High = 3# Very High = k. 

The average importance ratings as well as the percentage indicating monthly 
or more frequent usage are shown in table III-5* 

The data indicate that the library ranks third in terms of importance 
and fifth in terms of utilization frequency. The data indicate that the 
library plays a major role in fulfilling information needs. The only 
sources it does not surpass in importance are those more or less unavoidable 
sources which have a clear accessibility advantage over the library. 

The importance ratings of several sources are low, as shown in 
table III-5. It should be noted that the average ratings include frequent 

^ The data are not strictly comparable with those of the critical 
incident data presented earlier. The critical incident instructions asked 
respondents to exclude instances when the information came from someone in 
the immediate work group or from documents routinely used in everyday work. 



users, infrequent users, and non-users of each service. For this reason 
the low importance rankings are not felt to ; be significant. 



TABLE III-5-- UTILIZATION FREQUENCY AND PERCEIVED IMPORTANCE 
OF INFORMATION SOURCES BY LANGLEY RESEARCH 
PROFESSIONALS 



t 



i-'. 




Information source 


Percent indicating 
use monthly or 
more often 


Average 

importance 

rating 


Ovm or colleague’s personal library 
or files 


89 


• 5-0 


Informal. discussions with members of 
own work group 


9° . 


2.9 


LRC library 


55 


2-5 


Informal discussions with other NASA 
employees 


70 


2.k 


Personally conducted experimentation 


43 


2.k- 


Documents/reports routinely distributed 
to my work group 


78 


2.8 


Peers in ny field in other organizations 


22 


1-9 


NASA meetings, seminars, presentations 
of papers 


20 


1-7 


Catalogs and documents supplied by 
vendors and contractors 


50 


1*7 


Vendor/contractor personnel 


55 


1*7 


Conventions/professional meetings, 
trade shows, symposia, etc. 


k 


1.2 


Paid university/industry consultants 


9 


1.0 


Other libraries 


7 


0-9 
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Information Sources - Administrative Professionals 



The sample of professional-level administrative personnel answered 
questions similar to those of the research personnel on importance and 
frequency of utilization of various information sources. The question was 
changed slightly by deleting "personally conducted experimentation" and 
adding "policy or procedure manuals, regulations, codes, etc." (See 
Questions 7 and 8, appendix B. ) The results are summarized in table III-6 
in the same manner as were data from the research professionals. 

The data clearly show that the library occupies a considerably lower 
position vis-a-vis other information sources for administrative profes- 
sionals than it does for research professionals. The library ranks ninth 
in importance and tenth in utilization frequency. The results are not 
surprising in view of the fact that the library is designed primarily to 
meet the needs of the research professionals. The data do demonstrate the 
fact that different jobs have different sets of information needs and 
therefore result in different patterns of information source utilization. 



TABLE III-6. - UTILIZATION FREQUENCY AND PERCEIVED IMPORTANCE 
OF INFORMATION SOURCES BY ADMINISTRATIVE 
PROFESSIONALS 



Information source 


Percent indicating 
use monthly or 
more often 


Average 

importance 

rating 


Policy or procedure manuals, 
regulations, codes, etc. 


76 


3-4 


Informal discussion with own work 
group 


86 


2.9 


Other NASA employees 


76 


2.7 


Reports and publications routinely 
distributed to my work group 


88 


2.4 


Own or colleague's personal library 
or files 


64 


2.3 


Peers in ray field in other 
organizations 


45 


2.2 


NASA meetings and seminars 


29 


2.1 


Vendor/contract of personnel 


47 


1.8 


Books, periodicals, or documents 
from LRC library 


28 


1.5 


Conventions, professional meetings, 
trade shows, etc. 


10 


1.5 


Catalogs and documents supplied by 
vendors or contractors 


35 


•1.3 


Other libraries 


14 


1.2 


Paid university/ industry consultants 


2 


1.0 



Comparative Library Utilization 



If it is true that different jots have different sets of information 
demands* then perhaps it would be of interest to examine library utilization 
by various subgroups within the total sample. The percent of library users 
(i.e. * those indicating use of the library more than six times in the last 
12 months) is shown in table III-7* 



TABhE III-7.- PERCENT OF VARIOUS SUBGROUPS CLASSIFIED AS 

LIBRARY USERS 





Number 


Percent 

users 


All research respondents 


319 


73 


All administrative respondents 


49 


23 


Research respondents 






Managers 


76 


69 


Non-managers-*- 


2kl 


75 


Less than 10 years at Langley 


164 


77 


More than 10 years at Langley 


155 


70 


Bachelor’s degrees 


207 


63 


Advanced degrees 


111 


92 


Research task 


125 


89 


Design/development task 


89 


44 


Engineer 


200 


72 


Scientist 


93 


74 


Analysis and Computation Division (ACD) 


23 


26 


Research divisions (excluding ACD) 


224 


86 


Engineering and technical services 


34 


32 


Project offices 


12 


42 



Sum of managers and non-managers does not equal total 
number of respondents (319) because a few respondents did not 
indicate their supervisory level. For similar reasons* other 
groupings may not add to 319. 
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Several points deserve comment. The very high utilization percentages 
demonstrated by research professionals with advanced degrees indicates the 
importance of experience in using a library. These individuals undoubtedly 
have had more experience with libraries and therefore tend to use it more. 

In addition, they are more likely to be engaged in research tasks as 
opposed to design/development tasks where they would have greater need to 
refer to the professional literature. 

The data by division are also interesting. It is not too surprising 
that the research divisions have a high percentage of library users. The 
low percentage utilization for the Analysis and Computation Division, 
the engineering and technical service divisions, and the project offices 
would appear to indicate that the information needs of their jobs are 
substantially different. Wolek ( 1969 * P* ^7 2 ) points out that the inform- 
ation needs of project personnel become more and more narrow as the design 
of their project becomes "frozen. " He suggests that it may be dysfunctional 
for project personnel to continue their information search: "it becomes 

more and more important that the engineer not be diverted to new techno- 
logical concepts or possibilities." 

Why Non-Users Are Non-Users 

All individuals indicating that they did not use the library more than 
six times in the preceding 12 months were asked to comment on why they did 
not use the library more frequently (see Question 23). The results of the 
responses to this question by the 8k non-users are summarized in table III-8. 
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TABLE III-8.- NON-USERS' REASONS FOR NON-USE OF LIBRARY* 



£ 



Response 


Percent 


Information needs of my job 




a. 


Are better or more easily fulfilled 
from other sources 


50 


b. 


Are sufficiently fulfilled by branch 
or personal library 


6 


c. 


Just doesn’t require use of library 


10 


I used the library more in the past but 




a* 


My job has changed (typically to 
management or project office) 


12 


b. 


I quit because of microfilm reading 
problems or books always being out 


5 


c. 


I’ve been on educational/military 
leave 


4 


The library does not fulfill my information 




needs because 




a. 


The collection does not contain 
information on the topics relevant 
to my job 


3 


b. 


The material I need is beyond state 
of the art and is thereby not yet 
in print 


5 


Other 






a. 


Too busy 


3 


b. 


Library too far away 


4 



*Based on responses from 84 respondents. 
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The responses clearly indicate that their reasons for non-use are 
mostly cases of non-need rather than any shortcoming on the part of the 
library collection or the services offered. 

This finding confirms the earlier analysis of data presented on 
library use by divisions and by job task, which suggested that job task 
is a major determinant of information needs and, therefore, of information 
sources utilized. 

It would seem reasonable to conclude that the library is used by all, 
or almost all, of those having information needs which can be met by the 
library. 
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SECTION IV 



EVALUATION OF LIBRARY COLLECTIONS, SERVICES, PERSONNEL, 

TOOLS, AND FACILITIES 

This portion of the report summarizes and analyzes the responses of 
the respondents categorized as users. (Question 22 (see Questionnaire, 
appendix A) asked if the respondent had used the library services, includ- 
ing both visits to the library and telephone or written requests for 
services, more than six times in the last 12 months. Those -who answered 
yes (251 or 73 percent of those responding) were asked to complete a series 
of questions (Questions 2k - 35 ), relating to the library collections, 
services, and facilities. The results reported in this part of the report 
are, with a few exceptions, a summary of responses made by those library 
users . 

The Collection 

Respondents were asked how often they used various portions of the 
total collection (Question 24). Their responses are summarized in 
table IV-1 which gives the percentage of respondents indicating use weekly 
or more often, and those indicating use monthly or more often. It can be 
seen that the paper copy collection of documents and reports are most 
frequently used, followed by periodicals and microfiche documents. 
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TABLE IV-1. - UTILIZATION AND USEFULNESS RATING OF VARIOUS 
PORTIONS OF LIBRARY COLLECTION 



Collection 


Percent reporting use 


Usefulness 

rating 


Weekly or 
more often 


Monthly or 
more often 


Paper copy collection of reports 


31 


55 


2.75 


Periodicals (current and bound) 


22 


48 


2.50 


Microfiche reports 


20 


49 


1.79 


Books 


19 


42 


2.33 


Classified documents 


7 


17 


1.30 



Question 25 asked users to rank the various portions of the collection 
with respect to the usefulness to their work by selecting one of five 
choices as follows: Very Low, Low, Moderate, High, Very High. Responses 

were scored by assigning numerical values to each response with a 
Very Low = O, Low = 1, Moderate = 2, High = 3> and Very High =4. The 
average usefulness rating of each portion of the collection is also shown 
in table IV-1. 

It is noteworthy that the microfiche collection ranks third in util- 
ization and fourth in usefulness in view of its size vis-a-vis other 
portions of the collection. The microfiche collection is larger than the 
paper copy collection of documents, yet ranks below it in utilization fre- 
quency and well below it in usefulness. 

The low utilization frequency of classified documents is not surprising 
in view of -the "need to know" criteria. Since the average usefulness rating 
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includes ratings of many respondents who simply do not utilize classified 
documents, its low value is not felt to be significant. 

Question 2 6 asked respondents to name a particular field or fields in 
which the library collection has made a significant contribution to your 
work because of its strength or depth. A total of 150 responses were 
received to this question ('some respondents contributed more than one). 

The answers were classified in various categories. The results of this 
classification are shown in table IV-2. 

TABLE IV-2. - FIELDS IN WHICH USERS INDICATE LIBRARY HAS 
SIGNIFICANTLY HELPED WORK 



Field 


Number of 
responses 


Aircraft and aerodynamics 


30 


Fluid mechanics, gas dynamics 


29 


Mathematics, statistics, computers 


20 


Materials and structures 


Ik 


Heat transfer 


11 


Chemistry 


10 


Physics (acoustics, czyogenics, 
geophysics) 


10 


Space flight/orbital mechanics 


6 


Other (ranging from pyrotechnics 
to electron microscopes) 


20 



3 ^ 
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Question 27 asked users if there were any particular fields in which 
the library collection would, if strengthened, noticeably help their work. 
A total of 47 responses were received (some respondents contributed more 
than one). 

TABLE IV-3.- FIELD IN WHICH RESPONDENTS INDICATE THEIR WORK 
WOULD BE NOTICEABLY HELPED IF LIBRARY COLLECTION 
WAS IMPROVED 



Field 


Number of 
responses 


Mathematics, statistics, computers 


7 


Aircraft, aerodynamics 


5 


Physics 


5 


Chemistry 


b 


Instrumentation 


b 


Management, psychology 


b 


Biology 


3 


Other, unclassifiable 


21 




b 7 



Respondents indicate about three times as many fields in which they 
have been helped by the collection as they indicate fields that would 
help if strengthened. Many of those responding to Question 27 repeated 
their answer given in Question 26. They were saying that the collection 
had helped them, but that further strength would help them more. 

Five respondents used this question to suggest the collection could be 
improved if more back issues of journals and more books (specifically text- 
books) were added. 
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There were several opportunities for users to make comments on the 
library in general (Question 35 ) or to relate difficulties they had 
experienced utilizing the library (Question 30) or to suggest improvements 
(Questions 31 and 32)* Only four of some 160 responses to the "difficulty" 
question pertained bo the collection. The difficulties resulted from 
reports not being available. A few (7 of 200) responses to the general 
comments question pertained to the collection. All but one indicated the 
collection was adequate to very good. .A few suggested improvements per- 
tained to the collection. These all suggested expansion of the periodical 
collection, primarily through the acquisition of back issues of journals. 

The low level of responses to all these questions indicates that the 
collection is adequately fulfilling the information needs of research 
professionals. Now, let us examine the situation for the administrative 
professionals. 

Response from Administrative Professionals 

The results from the questionnaire sent to the administrative pro- 
fessionals did not inquire about usage of particular portions of the 
collections since it was felt that most of these personnel are infrequent 
users of the library. However, administrative professionals were asked the 
same questions about fields in which the library had helped their work or 
would help if the collection was strengthened. From the k9 respondents, 

10 fields were indicated in whicn the library had helped. Twenty-one 
responses were obtained on the "stronger collections would help" question. 
The results are summarized in table IV-4. 



TABLE IV -4.- FIELDS WHICH ADMINISTRATIVE PERSONNEL INDICATE 
LIBRARY COLLECTIONS HAVE HELPED OR COULD HELP 
THEIR WORK 



Field 


Have made a 
contribution 


Would help 
if strengthened 


Mathematics, statistics, computers 


4 


2 


Management 


1 


10 


Cost accounting, pricing 


- 


3 


Other, miscellaneous 




_6 




10 


21 



Note that the ratio of "have made a contribution" responses to the 
"stronger collections would help" responses is about 1:2 in the case of 
administrative professionals. This is in sharp contrast to the 3:1 ratio 
observed among research professionals. 
library Tools and Facilities 

In this section, survey data will be presented on the frequency of 
utilization and importance ratings of various library tools and facilities. 
Following this comparison of all the tools and facilities, additional, survey 
data are presented for individual facilities where appropriate. 

Users were asked the frequency with which they utilized various 
library tools and facilities in Question 28. They were also asked to 
indicate the degree of importance to their work of each (Question 29)* 

The degree of importance ratings were scored as before (i.e.. Very Low = 0, 
Low = 1, Moderate = 2, and Very High = 4). Frequency of utilization 
responses are summarized in table TV-5* 



TABLE IV-5-- UTILIZATION AND IMPORTANCE RATINGS 
OF LIBRARY TOOLS AND FACILITIES 



Tool/facility 


Percent reporting use 


Average 


Monthly or 
more often 


Few times/yr 
or more often 


importance 

rating 


Abstracting services (STAR, IAA, 
etc. ) 


57 


86 


2.30 


KWIC-combined index printout 


7 


57 


0.80 


Card catalog (books ) 


19 


81 


1.74 


Card catalog (pre-1962 documents) 


11 


60 


1-73 


Microfilm reader 


17 


65 


1.12 


RECON 


1 


33 


0.86 



The one tool that stands out in both utilization frequency and importance 
is the abstracting services. This is not surprising in view of the wide 
circulation of these publications at Langley. 

The low usage of the microfilm reader is somewhat surprising. It may be 
that respondents interpreted this question to mean use of a reader at the 
library . If so, the low utilization is not surprising. 

The remaining tools are ones which users must come to the library to use. 
Since most users do not visit frequently (hut rather call or send a note), 
the low frequency of utilization and low importance ratings are no^fc sur- 
prising. Furthermore, the user (as we shall see later) makes use of library 
personnel to find the information he seeks rather than use of the tools 
himself. The personnel are more accessible and a lot more pleasant to deal 



Evaluation of Individual Library Tools 



Abstract/Announcement Publications - One of the principal ways in 
which patrons find out what is new in particular fields is through announce- 
ment services. Four such services announce for the entire NASA system. 

The services are as follows: 

Scientific and Technical Aerospace Reports (STAR) 

International Aerospace Abstracts (IAA) 

Classified Scientific and Technical Aerospace Reports (CSTAR) 

Selected Current Aerospace Notices (SCAN) 

The first three of the above are abstract services covering (l) reports 
(STAR), (2) boohs, journals, conference papers (IAA), and ( 3 ) classified 
reports (CSTAR). The fourth is an announcement service only designed for 
selective dissemination to individuals by topic. 

All respondents were asked about their use of or familiarity with the 
four announcement services. This is an exception to the general practice 
followed in Section I V of reporting on responses of those individuals who 
are classified as library users. The data are summarized in table I V-6. 

TABLE IV-6.- ALL RESPONDENTS FAMILIARITY WITH AND USE OF 
NASA ANNOUNCEMENT SERVICES 



Service 


Percentage 


Have 

used 


Not used 
but heard of 


Never 
heard of 


STAR 


77 


11 


12 


CSTAR 


5^ 


20 


26 


IAA 


5^ 


17 


29 


SCAN 


25 


2k 


51 



0 




45 
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The responses indicate that the services are widely used. Somewhat 
surprising, however, is the relatively low use and recognition of SCAN. 

Less than half of all the respondents have heard of it, and only half of 
those use it. In view of the fact that three-fourths of the respondents 
have used STAR, and over half have used CSTAR and IAA, it may well be that 
SCAN is redundant in that it covers the same reports and publications as 
do the older and more established -abstract services. 

There were several open-ended questions in the questionnaire which 
ashed. users to comment on many difficulties they had ha.d using the library 
(Question JO), improvements they would suggest (Questions J1 and J2). 

Question J5 asked for any comments of any nature they had about the 
library. Only three out of some 160 responses to the "difficulties" 
question pertained to abstracts or announcement services, thus indicating 
that these services axe one of the stronger parts of the total information 
system. The few complaints made were that the abstracts were "vague and, 
in some cases, misleading, in that authors used new nomenclature to txy to 
make some old findings appear new. Suggested improvements were to have 
more skilled personnel doing the abstracting. 

The wide usage of and the general lack of difficulty reported lead to 
the conclusion that the abstracting/announcement services are fulfilling 
their function well. 

Microfiche - The survey clearly reveals a widespread and strong dislike 
for microfiche. In an open-ended question, "difficulties in utilizing the 
libraxy, " 79 of the l6l comments (or lj-9 percent) pertained to microfiche, 
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readers, or copies made from microfiche. No other categoiy received more 
than 15 percent of the total difficulty responses. A few comments of 
respondents follow: 

Respondent 8 : "Have to search building for microfilm reader, carry 

it up or down a couple of flights of stairs j it takes 
up a large area on the desk, is awkward to use and 
hard on the eyes. " 

Respondent 30 . "I hate microfilms and will use them only as a last 
resort . " 

Respondent 13k : "Microfilm reports seem to get thrown in a drawer for 
future perusal, then do not get read." 

Respondent 22 : ". . . tend to put off looking at it." 

Respondent 251 : "How is one supposed to refer from text to tables 
using one of those readers?" 

Respondent 56 : "Microfilm enlargements are sometimes unreadable and 
always curl up . . . " 

It is obvious that the opportunities for frustration are numerous in 
using microfiche. Once film is obtained, there may be a problem in 
(l) reader accessibility, ( 2 ) illegible original document and, consequently, 
illegible film, ( 3 ) poor quality reproduction on film, (Ij-) awkwardness in 
using reader, particularly in referring from text to tables, ( 5 ) poor 
quality reader (or reader out of adjustment), and (6) inability to get 
legible paper copy of material found to require extensive study. 

There is no question that the quality of readers and reproducing 
equipment has improved in recent years. However, it is equally clear that 
the problems associated with the use of microfiche have not been solved. 

This is not a local complaint about a local issue. Robert B. Thrall, a 
prominent scholar and past editor of Management Science rejects the use of 
microfilm: "With microfiche the entire contents of a journal can be 



recorded on film the size of an index card. However, there is as yet no 
generally satisfactory reading device . " (l970> P* B- 58 O. ) Were difficulties 

in utilizing microfiche only an inconvenience, it would not he of great 
concern. But as several of the above comments by respondents indicate, 
it is not only an inconvenience, but it is a deterrent. 

RECON - The earlier data (table IV-5) indicated relatively little usage 
of RECON, the remote console which provides access to the computer stored 
index of reports and publications at College Park. In view of the low usage 
(only 1 percent indicate monthly or more frequent usage), it is somewhat 
surprising that 8 , or 5 percent, of the responses to the "difficulty" 
question pertain to RECON. Of these complaints, four were that RECON was 
too slow and three pertained to RECON’s not working (i.e. , down) when needed. 

The author is, by virtue of his limited observations and experiences, 
in agreement with those reporting difficulties. Of the two terminals at 
the library, only one was operable at any time during June and July in spite 
of one yisit by a service engineer. The remaining terminal was frequently 
inoperable because the central computer at College Park was down. Even 
when operating, delays of 1 minute were frequent before the computer would 
accept an instruction. Perhaps this writer has been oversold on computer 
capability by advocates to whom a nanosecond is a long time. However, for 
a system that is a major part of such a massive information retrieval system, 
RECON seems unnecessarily unreliable and slow. 

Library Services 

The discussion of library services and personnel follows the pattern 
used before, namely, presentation of data on utilization frequency and 
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important rankings of all the services, followed fey further analysis of 
individual services. Responses to Questions 28 and 29 which inquired into 
utilization and importance of library services are presented in table IV-7 • 



TABLE IV-7. - UTILIZATION AND IMPORTANCE RATINGS OF LIBRARY SERVICES 



Service 


Percent reporting use 


Average 


Monthly or 
more often 


Few times/yr 
or more often 


importance 

rating 


Help by library personnel in locating 
specific book or report 


42 


97 


2.70 


Literature searching by library 
personnel to find information 
for which I had no specific lead 


10 


82 


2.38 


Help by library personnel in use of 
"finding tools" 


10 


80 


2.05 


Copy service 


47 


97 


2.62 


Checkout service 


61 


99 


2.75 



Note that "help by library personnel in finding a specific report or 
book" is used monthly by 42 percent of* the users, and several times per year 
by nearly all (97 percent). Comparison of these data with that on utiliza- 
tion of the finding tools (table IV-5 ) (KWIC index, card catalog, etc. ) 
indicates that most users rely much more heavily on the library staff rather 
than doing it themselves.' 1 ' Confirming this conclusion are the importance 
ratings. The importance rating for reference service is 2.70, while that 
of the finding tools ranges from 0.80 to 1.74^ (see table IV-5). 

%ith the exception of using the abstracts which probably are used 
more as an announcement service than as a finding tool. 
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Literature searching 'by library personnel was used by 10 percent of 
•the library users monthly and by 80 percent occasionally. The high impor- 
tance rating of 2.38 given, this service is phenomenal, considering the 
relatively low frequency of use (caused apparently by lack of supply rather 
than lack, of demand). For example, the widely used (57 percent report 
monthly or more frequent use) abstracting services received an importance 
rating of 2.30, somewhat less than the literature search service for which 
only 10 percent report usage monthly or more often. 

Copy service and checkout service are both very heavily used. Nearly 
half of the respondents reported monthly or more frequent requests for 
copies, -while over 60 percent check something out monthly or more often. 
Virtually all (97 percent and 99 percent, respectively) use these services 
at least several times a year, which indicates that circulation (which 
includes copy give-aways) is the high-volume library operation. In terms 
of importance ratings, each of the circulation services rank essentially 
equal with reference services. 

With respect to all library services, the "difficulties" question 
resulted in a relatively low number associated with library services. Of 
these, the largest number were associated, as one would predict, with the 
most heavily used service, circulation. The difficulties reported which 
pertain to library service are summarized in table IV-8. 

It should be noted that almost half of those who reported difficulties 
associated, with slow copy service noted that the slow delivery was associated 
with requests for documents not available locally and had to be ordered from 



the "Facility. " The difficulty for these respondents (and perhaps for 
others who were not so specific) result from system design, not its 
operation. 

TABLE IV- 8 . - REPORTED DIFFICULTIES ASSOCIATED WITH 
LIBRARY SERVICES 



Difficulty 


Number of 
times reported 


Reference : 




Help not readily available 


5 


Couldn't find report requested 


1 

k 


Literature search: 




Wot readily available 


2 


Provided too much information 


2 

k 


Circulation : 




Slow copy service 


15 


Book charged erroneously 


1 


Wo follow-up on documents ordered from facility 


2 


Documents not available locally not ordered 


_1 


promptly 


17 



Considering the relatively low level of difficulties reported pertaining 
to library services (25, or 15. percent, of total "difficulty" responses),' it 
can be concluded that respondents find services adequate to meet their needs. 
However, the responses also indicate that the staff is being pushed to 
handle all the demands placed on them by a larger collection and a greater 
number of patrons. 



Accessibility /information About Library 

Throughout the preceding discussion, reference has often been made to 
Question 30 which asked users if they had experienced any difficulty in 
using the library. There were a number of responses to this question which 
do nob neatly fit into the pattern of organization adopted for this section. 
Almos-fc all of these responses can be grouped under the broad heading of 
accessibility. This includes accessibility to the collection itself, 
accessibility bo finding tools, and difficulties posed by lack of knowledge 
of what the library has to offer. 



TABLE IV -9. - REPORTED DIFFICULTIES ASSOCIATED WITH ACCESSIBILITY 



Difficulty 


Number of 
times reported. 


Books are checked out by others 


10 


Individuals having book are slow to return 


2 


when requested 


* 


Books I have are requested by others 


1 


Periodicals missing from shelves or otherwise 


k 


difficult to locate 




NASA reports not accessible 


2 


Book collection split (Dewey/LC). 


1 


Finding tools are inaccessible, difficult to use. 


8 


or incomplete 




Do not know how to use library or what it has 


_8 


to offer 


36 



9 
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These responses indicate some degree of difficulty by users. The 
frustrations caused when others have books out or periodicals have been 
sent to the bindery are perhaps unavoidable. However, responses in the 
last two categories above indicate that some users cannot get at or do not 
know how to use library finding tools or do not know what is available. 
Overall impressions 

A great deal of Section IV has dealt with problems or difficulties 
reported in utilizing the library. Respondents reported difficulties 
because they were specifically asked to do so. They were not asked to 
report their successes or their favorable opinions. The questionnaire was 
not designed to elicit compliments. Thus, it is impressive that most users 
did comment favorably in response to Question 35 which was "Do you have any 
comments about the library, its collection, services, facilities, personnel 
etc.?" A total of 197 responses (some respondents contributed more than 
one) were received. They are summarized in table IV-10. 

TABLE IV-10. - CONTENT ANALYSIS OF RESPONSES TO GENERAL COMMENTS QUESTION 



General nature of response 


Number of 
times reported 


Personnel competent, friendly, cooperative, 
helpful, efficient 


96 


Library, in general, is best, outstanding, 
essential, excellent 




Library services are excellent, complete, 
satisfactory, adequate, timely 


21 


Library collection is adequate, good 


6 


Total negative comments 






197 



The negative comments appeared to be roughly distributed in the same 
proportion as in Question JO, which inquired about difficulties. No new 
issues were raised. 



Although the proportion of favorable to unfavorable comments is 
impressive, it cannot capture the real flavor of the appreciation, gratitude, 
and compliments expressed by the users. A few of the respondents’ comments 
may help. 

Respondent k- 7 ’ "Would be lost without it. " 

Respondent 226 : "Most of my trips to the library would end in 

frustration were it not for the excellent service 
rendered by the staff. These people should be 
commended for the excellent service they render. " 



Respondent 56 : 

Respondent 76 : 
Respondent 88 : 
Respondent JJ : 

Respondent 5^ : 



". . . do an excellent job generally, in light of 
limited resources." 

"Better than could be expected. " 

"Nothing but ridiculously good service." 

"They do a wonderful job with what they have to work 
with . . . research would be impossible without their 
generous cooperation. " 

"In my thinking, the library is the most complete 
facility, the most cooperative workers, and offers 
the best services available an;/where. " 



And finally, from a respondent whose questionnaire came too late to be 
included in most of the analysis: "I think a special point should be made 

of* the apparent interest that library employees have in helping us solve our 
problem. This is pleasantly at variance to the frequently encountered 



position that a job entails discharge of certain minimal responsibilities. 
Their attitude is refreshing. " 
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SECTION V 



CONCLUSIONS AND RECOMMENDATIONS 

"it’s not enough to design a system 
that can be used. A system has to 
be designed -which will motivate people 
to use it." William E. Moffett 

The major conclusion of this report is that the LRC library plays a 
vital and important role in supplying Langley research professionals with 
technical and scientific information. This is evidenced by the comparison 
of sources of useful information of Langley scientists and engineers with 
those of similar personnel in the research divisions of several large 
corporations. Langley professionals with similar-type jobs use the type 
of literature furnished by the library more than do comparable professionals 
in other organizations. Further reinforcement of this conclusion is found 
in the high frequency of usage and the high importance rankings of the 
library shown by the survey of research professionals. Analysis of use by 
job type and division further indicates that all, or nearly all, of the 
engineers and scientists having jobs with information needs that can be met 
with written published documents are utilizing the library. In short, the 
library plays a major role in providing essential information to its users, 
who are numerous. 

In the remaining portion of the report, the author will discuss some 
of the findings with an emphasis on improvements which should be made. While 
this approach is critical in some areas, it should be emphasized that the 
overwhelming weight of evidence and observation leads inevitably to the 
conclusion that the library is an outstanding one. 
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Size Versus Accessibility 



If, as Gerstberger and Allen (1968, p. 279) have shown, perceived 
accessibility is the primaiy criteria by which information sources are 
selected, then making information accessible is as important as collecting 
it, if not more important. 

What this means to the library, or any organization which would serve 
as a source of infoimation, is that a small, highly accessible collection 
may provide more information to the user than a large, relatively inacces- 
sible collection. This is not a recommendation to reduce the size of the 
collection. Rather, its purpose is to make the point that there is a 
trade-off between the size of a collection and the accessibility to that 
collection. 

The second major conclusion of this report is that resources have been 
far more readily committed to increasing the size of the collection than to 
increasing its accessibility. This has resulted in an imbalance - a magnifi- 
cent collection with some deterrents to its use because of accessibility. 

Accessibility is where the problems are. Alm ost all the difficulties 
experienced by respondents pertain to accessibility. That is what complaints 
on microfiche, slow copy service, reference help not being available, books 
are checked out by someone else, etc., are all about. There are several 
other ways in which the data reinforce this point. 

A vivid example of the effect of accessibility is seen in the utiliza- 
tion frequency reports in different form. About 120,000 reports are avail- 
able locally in paper copy, while about 2^0,000 are available only on 
microfiche. Respondents to this survey have strongly indicated that 
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information on microfiche is not as accessible as information on paper. 
Respondents indicated that they used the paper copy report collection more 
often and it was more useful than the microfiche collection. Circulation 
figures bear this out. Paper copy report circulation averages 1,700 per 
month, while microfiche averages 1,100. Each paper copy document gets 
roughly three times the circulation of each microfiche document . Further- 
more, if we are to believe what many respondents say about microfilm, much 
of it gets filed for future reading and stays filed. 

It is certainly true that the local collection of reports includes 
many which are done locally or done on contract for LRC. Therefore, one 
would expect somewhat higher utilization frequency of the local collection 
because of higher interest. However, it seems doubtful to this writer that 
.local reports have that great an interest advantage over other reports. 

Another example of the importance of accessibility can be found in 
the changes in patterns of circulation that have occurred since Recall 

that while book circulation has remained roughly constant, periodical circu- 
lation has grown dramatically from 13 to 33 percent of the total. During 
the same 6-year period, the report circulation has dropped from 77 percent 
of the total to 56 percent. 

It is clear that' accessibility has increased for periodicals and 
decreased for reports during this period as can be seen in the following: 

1. Few periodicals were cataloged prior to the step to centralize 
cataloging in College Park. Since that time, articles selected for abstract- 
ing by IAA are included in the master index. This places these selected 
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articles in the KWIC index and on RECON in addition to announcing them in 
the widely distributed IAA abstract series. 

2. Most of the articles abstracted (and therefore included in the 
system) are available in the library. 

3* Xerox has become more available in recent years. 

4. Reports, while announced in STAR and included in the indices and 
RECON, have gained little over their earlier accessibility since they were 
cataloged at LRC. 

5* Only one-third of the reports are available on paper copy. The 
remainder are on microfiche. Formerly, almost all reports were available 
locally in paper copy form. 

6. Paper copy from microfiche is (l) less legible than Xerox, 

(2) not available approximately half the time because the machine which 
produces paper copy from microfiche is not operable, and ( 3 ) requires 
section head approval to obtain. 

Once again, it appears that accessibility is a more important variable 
than size in determining use of information. 

Recommendations 

Since accessibility is so highly important, the recommendations which 
follow are directed toward improving accessibility. The reader should not 
lose sight of the fact that the high use made of the library indicates that 
its services are generally accessible - certainly more so than the average 
library. The recommendations which follow are aimed at improving an already 
generally high level of accessibility. Recommendations are divided into 
three major categories as follows: 
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1. Facility related - pertain to service rendered "by College Park 
Facility. 

2. Internal changes at LRC library. 

3* Support required for continued excellence. 

The major factors reducing the accessibility of reports are the myriad 
of problems associated with the use of microfiche. 

In view of the fact that microfiche reports simply are not utilized, 
because of their form, serious investigation of alternate systems should 
be undertaken. While detailed feasibility analysis is beyond the scope of 
this report, several ideas should be investigated such as the following: 

Obtain one printed copy of each report rather than microfilm. Xerox 
copies of the original as needed. There are several variations that should 
be explored such as relying on film for all reports over 5 years old in 
order to minimize storage problems. Another alternate 'would be to rely 
; largely on film, but order an original for all reports demanded one or 

more times in the first year after announcement in STAR. In this way the 
patrons select the most important items which can then be made available 
in more accessible form for future use. 

Another possibility is to get completely out of the film business, 
utilizing facsimile transmission from the facility for any report requested. 
This equipment may be too slow and/or expensive for the volume involved. 

It may be possible to utilize RECON transmission lines for the facsimile 
transmission during off hours to provide an overnight service. An even 
simpler approach is to offer the user a choice of microfilm today or a 
Xerox from the facility in 2 days to 5 days. 
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If none of the above proves practical, then the minimum the facility 
should undertake is (l) to obtain from the author or type from copy the 
author supplies a legible copy of every document which is announced in 
STAR, (2) institute a quality control program on film reproduction. To 
announce and abstract a report for which a legible copy is not available 
converts an information retrieval service into a frustration retrieval 
service. Frustrated users tend to become non-users. 

Langley Library 

There are numerous areas in which the library can improve the accessi- 
bility of its collection as follows: 

Microfiche : If a viable alternate to use of microfiche cannot be found 

and the facility does not improve quality of the film, then the library 
should institute its own quality control program. Poor quality film should 
be returned to the facility. 

If film is sent to patrons, a printed note should be 
attached that says something like "if you find after examination that this 
report is of significance and you need to study it in detail, please request 
a hard copy. " It appears that a great number of users do not understand 
that hard copy can be made available. The requirement for section head 
approval should be dropped. 

The library should continue its investigation of better 
reproducing equipment and pressure the manufacturer of the present equipment 
for better maintenance, leading to a higher service factor and better 
reproduction. 
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Finding Tools ; The library needs to improve its finding tools and 



make them more accessible to users. Many patrons will not use them, but 
some would rather "do it themselves. " To the extent that patrons serve 
themselves, the limited library staff can devote to providing better service 
for those who request it. The following suggestions are made to accomplish 
this aim: 

1. The card catalog for books should be completed by adding the 
missing subject entries. 

2. The card catalog for pre -1963 reports should be made accessible. 
This would require removing classified material. 

3* A current index of periodicals should be provided in the reading 

room. 

4. The KWIC indices should be made accessible to patrons. If legible 
films of these indices can be provided and some projection device found 
which will reproduce a quality image, such a system should be promoted.* 1 * 

Collections : The library collection should be made more accessible 

by the following steps: 

Placing a due date on items checked out. It is estimated 
that the average book checked out stays out for 40 months. It seems to this 
writer that either the patrons are (l) utilizing the book regularly in their 
work, in which case they should purchase the book through their division, 
or (2) only keeping the book because of possible future usefulness. In 

^"There is no real contradiction in suggesting the use of film. The 
present form -and voluminous size of these indices make them difficult to use. 
A quality film-based system could make them more accessible. The same is 
not true of a typical report. 



this case, the book should be returned to the library so that others might 
be able to use it. It would seem that a 6-month period, with a 6-month 
renewal option, would be a reasonable compromise between the present policy 
and the relatively short loan allowed by many libraries. 

The library collection would be more accessible if all 
the books were classified under the Library of Congress system rather than 
the present split system in which the books are divided between a Dewey 
Classification and the Library of Congress Classification. 

The Patrons 

The point was made early in this report that the library of 1970 was 
quite different in terms of size, information form, finding tools, etc., 
from the library of a decade ago. Yet there is no way for the relatively 
infrequent user or the non -user to know that. It also seems likely that 
many of the frequent users are not fully aware of the collections and 
services that are available. The library has been too modest. It should 
advertise. The following is a partial list of ideas which might be used 
to increase patron awareness. 

A users guide which describes the collections, the tools, and the 
services available. Give-away copies could be placed at the reference desk 
and in the reading room by the elevator and sent to new employees. 

A "reacquaintance tour" in which a division or branch is invited to 
the libraiy, given a brief tour, a cup of coffee, and a 5-minute talk 
describing the not-so-visible services and collections that are available. 

A series of short articles in the Langley Researcher describing 
individual services offered by the library. 



Directions for patrons in the form of signs or labels pointing out 
that (l) recent past issues of periodicals are available and they may be 
obtained by . . ., ( 2 ) this card catalog contains only references to 
books, (5) the books are divided into two sections: Dewey, covering books 

purchased before 1965$ and Library of Congress, covering all later 
purchases . 

Support Needed for Continued Excellence 

It is probably true that members of the library staff have, at one 
time or another, made all the recommendations that this report makes, along 
with some better recommendations. 

However, because of reduced budgets, manpower reductions, and the 
extreme difficulty of measuring the value, of a library's services, the 
library has found itself in a very low-priority situation. As a result 
it has made the best of split systems, tools of low utility, and reduced 
staff in the face of the data explosion and an increased number of patrons. 
One often cited Example is the cataloging system. In place of a locally 
compiled card catalog, a master tape of reports was made available from the 
facility. Using this tape and locally written computer programs, it is 
possible to obtain indices of the system-wide collection of reports and 

1 

articles by author, subject, or citation number. The system suffers a 
major disadvantage vis-a-vis a card catalog in that it cannot be easily 
updated by adding new cards. The program must be re-run and new index 
produced. Because of a shortage of computer time, the library's primary 
reference tool is often not current and is always split into four or five 
sections according to the date of initial citation. Thus, the librarian 
must look in four or five places instead of one. That takes longer. 



In that and in many ways, the "new" library requires more time and 
more skill on the part of its staff. Many of the difficulties experienced 
by survey respondents reflect the strain placed on the staff and the services 
they render in a larger library with less than adequate support. Continued 
lack of support will be reflected in reduced accessibility of information 
to Langley professionals. 

Quo Vadis 

It is undoubtedly true that the data explosion will continue. This 
will place additional strains on the library in future years. There appear 
to be no good solutions to the problems created by the data explosion. 
Ideally, a more critical screening process for getting reports or articles 
into the system would reduce the total volume of information and make it 
more manageable. However, such a step would require, literally, an army 
of highly trained and skilled judges. 

Schemes to discard data after a certain time period may have some 
merit. However, the high use of the pre-1963 card catalog of reports 
indicated in the survey and the numerous requests for back issues of 
journals (as far back as the turn of the century, in fact) indicate that 
such a procedure would frustrate many patrons. 

Several ideas which may be of value in helping to cope with the 
continuing deluge of data follow: 

1 . Rely more on periodical literature. This would follow the trend 
of utilization by patrons in recent years. Periodicals, at least those 
with editorial boards, do some screening for quality, uniqueness, and 
relevance. 
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2 . Make use of gatekeepers. Perhaps the library of the future will 
concentrate on really supplying a great deal of service to relatively few 
people and rely on the "gatekeeper , who is naturally inclined to do so 
anyway, to serve as a conduit to others. Both Hall (l969> P* U) and 
Allen (1969; P* l8) suggest this approach. 

3 . Increased aid in searching for information. One of the most 
highly rated services of the library is the literature search service. It 
would seem that this service would become even more useful in the future, 
particularly if searchers can sort the wheat from the chaff as well as 
locating articles and reports. In this instance, the literature searcher 
becomes a form of gatekeeper, operating in the library instead of the 
research division. 
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LANGLEY RESEARCH CENTER 

Memorandum 



to : date: July 1 , 1970 

FROM : Assistant Director for Administration 

SUBJECT: Participation in Technical Information/ Library 

Utilization Survey 



The attached questionnaire inquires into the way in which profes- 
sional employees at Langley Research Center find and use techni- 
cal information. It is concerned in particular with the role 
played by the LRC Technical Library in fulfilling information 
needs. The questionnaire is part of a study being conducted by 
Dr. H. D. Dewhir st who is at LRC on the Summer NASA-ASEE 
Research Fellowship Program. 

The purpose of the study is to provide a better understanding of 
the use of various technical information channels and to evaluate 
the performance of the LRC Library in providing useful infor- 
mation services. 



You may identify yourself although there is no requirement that 
you do so. No attempt will be made to identify individuals who 
do not choose to identify themselves. The questionnaire results 
will be processed by Dr. Dewhirst, analyzed, and reported only 
in aggregate statistical terms. 

Your cooperation in completing and returning the questionnaire 
promptly to Dr. Dewhirst at Mail Stop 109 will be appreciated. 




M/S 111 2741 

Enclosure 



NASA Langley Form 94 (June 1969) 
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TECHNICAL INFOEMATION/LIERARY SURVEY 
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Listed below are numerous possible sources of technical information which you may, or may not utilize in your 

work. Two distinct questions are asked about each source. 
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Paid university or industrial 
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Listed below are several different portions of the total library collects on. 

24. Please indicate the approximate 25. Please indicate the usefulness of each portion 

frequency with which you use each portion. to your work. 
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30. Please describe the nature of any difficulties you have experienced in utilizing any of the above 

services, tools, or facilities: 
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